Image based rendering allows the synthesis of a virtual viewpoint from a collection of camera images. For example, in an arrangement where a subject is surrounded by a ring of cameras, it is known that a new (virtual camera) view of the subject, corresponding to a position in between (real camera) captured views, may be synthesised from captured views if there is sufficient knowledge of the camera configuration and the scene.
In recent times, the ability to synthesise an arbitrary viewpoint has been promoted for the purpose of “free viewpoint” video. In free viewpoint video, the viewer is able to actively adjust the camera viewpoint to his or her preference within the constraints of the capture system.
Alternatively, sports broadcasters may make use of the synthesis of a virtual viewpoint to enable virtual cameras to broadcast perspectives not possible with physical cameras. Virtual cameras may totally replace physical cameras, or the virtual cameras may be incorporated into existing workflows to supplement physical cameras. As with physical cameras, camera operators control virtual cameras to produce a video stream that is fed to a producer/director's console and is then broadcast to viewers. Therefore, the video stream produced from a virtual camera is of comparable quality to the video stream produced by a physical camera.
Control of a camera, whether physical or virtual, to produce broadcast quality video is extremely challenging. An artistic eye for framing, good hand-eye coordination and fine motor skills are required of operators. Even with these skills, it may be difficult to consistently produce high quality video. A ‘Spidercam’ camera system is a camera system consisting of a camera rig suspended on wires, the rig being able to move in three dimensions within a limited area above the play of field. A Spidercam camera system is constrained by the requirement for physical wires to not interfere with the state of play. Therefore, a Spidercam camera system could not for example be used to show a players eye view of an arena. In the Spidercam camera system, a camera is mounted to a rig and is able to pan 360 degrees, tilt, and zoom. Spidercam camera systems require two operators, such is the difficulty of producing high quality video. One operator, a pilot, controls the position of the rig (i.e. the x,y,z co-ordinates of the camera within the space of the arena), the second camera operator controls the framing of the video using the orientation controls of pan, tilt, and zoom. The combination of position and orientation makes up the camera ‘pose’.
In a virtual camera environment, with free control, the camera may move in any of the three dimensions, while also panning, tilting, and zooming. A virtual camera is not constrained by physical properties and may literally be posed anywhere within the arena (i.e. position can change significantly almost instantly). Such a level of freedom is beyond even a ‘Spidercam’ camera system.
A need exists for a virtual camera which may be navigated in a manner that is intuitive to a user, while providing broadcast quality framing of events occurring within the arena.